Improving Models of Slipping, Guessing, and Moment-By-Moment Learning with Estimates of Skill Difficulty

Over the past several years, several extensions to Bayesian knowledge tracing have been proposed in order to improve predictions of students’ in-tutor and post-test performance. One such extension is Contextual Guess and Slip, which incorporates machine-learned models of students’ guess and slip behaviors in order to enhance the overall model’s predictive performance [Baker et al. 2008a]. Similar machine learning approaches have been introduced in order to detect specific problem-solving steps during which students most likely learned particular skills [Baker, Goldstein, and Heffernan in press]. However, one important class of features that have not been considered in machine learning models used in these two techniques is metrics of item and skill difficulty, a key type of feature in other assessment frameworks [e.g Hambleton, Swaminathan, & Rogers, 1991; Pavlik, Cen, & Koedinger 2009]. In this paper, a set of engineered features that quantify skill difficulty and related skill-level constructs are investigated in terms of their ability to improve models of guessing, slipping, and detecting moment-by-moment learning. Supervised machine learning models that have been trained using the new skill-difficulty features are compared to models from the original contextual guess and slip and moment-by-moment learning detector work. This includes performance comparisons for predicting students’ in-tutor responses, as well as post-test responses, for a pair of Cognitive Tutor data sets.